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DETAILED ACTION 
Response to Amendment 

1 . The amendment, filed 3/2/2006, has been entered and made of record. Claims 1-7,12- 
14,18-21,24,25,28,30,31,33-35 and 46-50 are pending in the application. 

2. Applicant's amendment to claim 5 has overcome the examiner 5 objection. 

Response to Arguments 

3. Applicant's arguments with respect to the claim 14 have been fully considered and are 
persuasive. Therefore, the Zhang et al. reference has been disqualified. However, upon further 
consideration, a new ground(s) of rejection is made (see below). 

4. Applicant's arguments filed 3/2/2005 with respect to claim 1 have been fully considered 
and are persuasive. However, a new grounds of rejection is not being made; only a new 
interpretation. In the previous office action, the examiner interpreted the light detecting elements 
as being the pixels on the image sensor device "303" of Hashimoto et al. In the new 
interpretation, the examiner interprets the light receiving elements as the pixels on the image 
sensor device "303" in Hashimoto et al. and the image capturing device for scene analysis "86" 
in Tamune. 

5. Applicant's arguments filed 3/2/2005 have been fully considered but they are not 
persuasive. 
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Throughout the Remarks Section, applicant asserts, "...Hashimoto arguably 
teaches. . .Ellenby arguably teaches. . .Cooper arguably teaches. . These arguments are not 
directed to the claimed subject matter. 

Specification 

6. The title of the invention is not descriptive. A new title is required that is clearly 
indicative of the invention to which the claims are directed. 

The following title is suggested: "IMAGE CAPTURING SYSTEM FOR CORRECTING 
COLOR BASED ON REFLECTING LIGHT FROM A SURFACE CONSTITUTED BY A 
MATERIAL WHICH FOLLOWS THE NEUTRAL INTERFACE REFLECTION THEORY". 

Claim Rejections - 35 USC § 103 

This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
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having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

7. Claims 1-7,12,13,19-21,24,25,28,30,31,47,49 and 50 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Hashimoto et al. (US # 6,072,526) in view of Tannine (US # 
6,940,556). 

As to claim 1, Hashimoto et al. teaches an image capturing system (Figure 3) for 
correction of colors in an image (Figure 3, amplifiers "Al" and "A2"), comprising: a camera 
(Figure 3) including a lens (Figure 3, lens "301"), an image capturing device (Figure 3, image 
sensor device "303"), light detecting elements (Figure 3, image sensor device "303"; {Pixels are 
inherent in image sensor devices. The light detecting elements are also constituted by pixels on 
the image capturing device "86" ofTamune which will be explained (see Response to 
Arguments, section 4) J), each of the image capturing devices and the light detecting elements 
having a plurality of color channels (Col. 5, Lines 64-67; {R and B channels are the color 
channels), a light-color measuring portion obtaining a value from one pixel or an average value 
from a plurality of pixels, for each of the color channels as reference signal values, out of light 
from a scene received by the light detecting elements (Figure 3, white balance signals "B" of 
"G/B" and "R" of "G/R" are the reference signals; Col. 5, Lines 64-67; Col. 6, Lines 1-40); and a 
correction unit (Figure 3, amplifiers "Al" and "A2") for correction of colors in the image by the 
reference signal values (Col. 6, Lines 40-43). The claim differs from Hashimoto et al. in that it 
further requires a reflection surface for capture of a main scene in the image capturing device and 
for capture in another image capture device; the reflection surface being disposed within a visual 
field of the camera for reflection of light from a reference scene disposed near the main scene for 
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reception by the light detecting elements via the lens and the reference signals being determined 
from the reference scene. 

In the same field of endeavor, Tamune teaches an electronic still camera (Figure 1) which 
employs two image capturing devices (Figure 1, image capturing device for photographing "73" 
and image capturing device for scene analysis "86"; {The reference scene is referred to as light 
entering the lens at an angle, and the main scene is light not entering the lens at an angle (i.e. 
light entering on the path shown in Figure 1. Both scenes are reflected by the quick return 
mirror.}). The first image-capturing device is used for photographing a main scene, and the 
second image-capturing device is used for scene analysis (color correction) (Col. 4, Lines 43-52; 
Col. 6, Lines 54-67). Light is sent to the scene analysis image-capturing device via a quick return 
mirror (Figure 1, quick return mirror "71"; {The examiner refers to the reflecting surface as the 
quick return mirror and the lens "301 " of HASHIMOTO et al}\ which reflects the incoming 
light to said image capturing device (Col. 4, Lines 38-47). In light of the teaching of Tamune, it 
would have been obvious to one of ordinary skill in the art to include the quick return mirror and 
second image-capturing device for the white balance calculation in Hashimoto et al. in parallel to 
the image sensor device "303" of Hashimoto et al., because an artisan of ordinary skill in the art 
would recognize that this would reduce the amount of time required for image processing which 
is implemented by the analyzing the photographic scene (see Tamune, Col. 1, Lines 56-60). 

As to claim 2, Hashimoto et al., as modified by Tamune, teaches the image capturing 
system according to claim 1, wherein the correction unit is a correcting portion for practical 
division by the reference signal values obtained for each of the color channels (see Hashimoto et 
al., Figure 3, amplifiers "Al", "A2"; {The gain for each amplifier is shown in Figure 3 (G/R and 
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G/B). Multiplying by G/R is equivalent to multiplying by G and dividing by R. The same applies 
for G/B.}), of respective main signal values at each of corresponding locations on coordinates in 
the main scene captured by the image capturing devices (see Hashimoto et al., Figure 3; {Main 
scene signals follow path to the input of Al andA2 (i.e. base of the triangle representing the 
amplifiers.)}), whereby obtaining corrected signal values as corrected values of the main signal 
value (see Hashimoto et al., Figure 3, R' and B' entering the color reproducing correcting 
circuit). 

As to claim 3, Hashimoto et al., as modified by Tamune, teaches the image processing 
unit used in the image capturing system according to claim 2, wherein coefficients (see 
Hashimoto et al., Figure 3, G/R and G/B) having the reference signal values as respective 
denominators are obtained in advance (see Tamune, Figure 5, Figure 7), for respective 
multiplication of these coefficients with the main signal values (see Hashimoto et al., Figure 3, 
G/R and G/B are gains), whereby performing correction of the main signal (see Hashimoto et al., 
Figure 3, R' and B' entering the color reproducing circuit). 

As to claim 4, Hashimoto et al., as modified by Tamune, teaches the image processing 
unit according to claim 3, wherein the coefficients have denominators respectively represented 
by the corresponding reference signal values (see Hashimoto et al., Figure 3, G/R and G/B), and 
a numerator represented by another coefficient(s) common to all of the color channels (see 
Hashimoto et al., Figure 3, G common to G/R and G/B). 

As to claim 5, Hashimoto et al., as modified by Tamune, teaches the image processing 
unit according to claim 4, wherein the coefficients are obtained from one frame of signals 
sequentially sent from the image capturing devices or the light detecting elements (see Tamune, 



Application/Control Number: 10/088,263 Page 7 

Art Unit: 2622 

separate image capturing device for scene analyzing "86" (one frame)), said coefficients being 
multiplied respectively with the main signal values obtained from another frame signal received 
at a later time (see Tamune, image capturing device for photographing "73" (another frame); 
Figure 5, Figure 7; full press- after half-press; Col. 4, Lines 38-55), whereby performing 
correction of the main signal (see Hashimoto et al., Figure 3, R' and B' entering the color 
reproducing circuit). 

As to claim 6, Hashimoto et al., as modified by Tamune, teaches the image processing 
unit according to claim 5, wherein the coefficients are multiplied respectively with a plurality of 
sets of the main signal values obtained from the plurality of other frames, whereby performing 
correction of the main signal (see Hashimoto et al., Col. 5, Lines 64-67; "...video output...). 

As to claim 7, Hashimoto et al., as modified by Tamune, teaches the image processing 
unit according to claim 5, further including a video amplifier for multiplication of the 
coefficients with the signals from the other frames (see Hashimoto et al., Figure 3, amplifiers 
"Al" and "A2"). 

As to claim 12, Hashimoto et al., as modified by Tamune, teaches the camera used in the 
image capturing system according to claim 1, further including a reflection surface moving 
mechanism capable of disposing the reflection surface out of the visual field of the camera (see 
Tamune, Figure 1; Col. 4, Lines 48-52). 

As to claim 13, Hashimoto et al, as modified by Tamune, teaches the image capturing 
system according to claim 1, further comprising a reflection surface moving mechanism capable 
of disposing the reflection surface out of the visual field of the camera for disposition of the 
reflection surface out of the visual field of the camera by the reflection surface after obtaining the 
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reference signal values for capture of the main image, the main signal values being corrected 
based on the reference signal values (see Tamune, Figure 5, Figure 7; Col 4, Lines 38-55). 

As to claim 19, Hashimoto et al., as modified by Tamune, teaches the camera used in the 
image capturing system according to claim 1, wherein each of the image capturing devices and 
the light detecting elements is constituted by an individual element of a same characteristic (All 
pixels on the image capturing devices produce a charge or voltage proportional to the amount of 
light incident). 

As to claim 20, Hashimoto et al., as modified by Tamune, teaches the camera according 
to claim 19, wherein the light detecting elements are part of the image capturing devices 
respectively (This inherent in the image capturing devices of Hashimoto et al and Tamune.). 

As to claim 21, Hashimoto et al., as modified by Tamune, teaches the camera used in the 
image capturing system according to claim 1, further including a storing portion for storage of an 
image file containing images captured in the image capturing devices or a holding portion for 
storage of a film recorded with said images, said images containing the reference image portion 
located at an end portion of an overall image region (It is inherent that the reference images are 
stored in the microcomputer when computing the G/B and G/R ratios. The end region of the 
overall image region is referred to as the scene analysis image capturing device.}). The claim 
differs from Hashimoto et al., as modified by Tamune, in that it further requires that the storage 
portion store the main scene images. 

In the same field of endeavor, Tamune teaches storing images from a main scene in a 
memory (Figure 1, camera internal memory "30"; Col. 5, Lines 21-29). In light of the teaching of 
Tamune, it would have been obvious to one of ordinary skill in the art to include a memory in the 
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camera of Hashimoto et al. 5 because an artisan of ordinary skill in the art would recognize that 
this would allow the user to review images taken prior to current use. 

As to claim 24, Hashimoto et al., as modified by Tamune, teaches the camera used in the 
image capturing system according to claim 1, wherein the main image is laterally elongated 
rectangular, the reference image portion being placed at an upper portion or a lower portion of 
the overall image region (see Hashimoto et al., Figure 3, image sensor device "303"; {A CCD 
device is either a square or a rectangle. In both cases, the image on the CCD is rectangular with 
laterally elongated sides. See claim 21 for overall image definition.}). 

As to claim 25, Hashimoto et al., as modified by Tamune, teaches the camera used in the 
image capturing system according to claim 1. Although Hashimoto et al., as modified by 
Tamune, does not state it explicitly, Official Notice is taken that the use of adjustable focus 
lenses and zoom lenses are well known and expected in the art (Applicant is directed to claim 1, 
where examiner defines reflection surface as quick return mirror of Tamune and lens "301 " of 
Hashimoto et al. Each time the focal length of the lens is changed, the angle and starting point of 
the reflection surface (partly lens "301 ") is changed.). One of ordinary skill in the art would 
have been motivated to use a focus adjustable lens and a zoom lens, because lenses of the sort 
have adjustable features, which help provide a user with the clearest, most natural looking 
picture. 

As to claim 28, Hashimoto et al., as modified by Tamune, teaches a recording medium 
recorded with software to be located into a computer for execution of the function realized by the 
image processing unit according to Claim 3 (see Hashimoto et al., CPU "314"). 
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As to claim 30, Hashimoto et al, as modified by Tamune, teaches the camera according 
to any one of claims 13 or 19 - 25. Although it is not stated explicitly, Official Notice is taken 
that the concept of using mechanical shutters to prevent light from entering an image sensing 
device is well-known and expected in the art. One of ordinary skill in the art would recognize 
that this is a quick, effective way to control exposure time. 

As to claim 31, Hashimoto et al. teaches an image capturing system (Figure 1) for 
stabilization of intensity in an image (Figure 1, amplifiers "Al", "A2"), comprising: a camera 
(Figure 3) including a lens (Figure 3, lens "301"), an image capturing device (Figure 3, image 
sensor device "303"), light detecting elements (Figure 3, image sensor device "303"; {Pixels are 
inherent in image sensor devices. The light detecting elements are also constituted by pixels on 
the image capturing device "86" of Tamune which will be explained (see Response to 
Arguments, section 4).})\ and an image processing unit obtaining a value from one pixel or an 
average value from a plurality of pixels, for each of the color channels as reference signal values, 
out of reflected light from a scene received by the light detecting elements (Figure 3, white 
balance signals "B" of "G/B" and "R" of "G/R" are the reference signals; Col. 5, Lines 64-67; 
Col. 6, Lines 1-40), for practical division by the reference signal values obtained for each of the 
color channels (see Hashimoto et al., Figure 3, amplifiers "Al", "A2"; {The gain for each 
amplifier is shown in Figure 3 (G/R and G/B). Multiplying by G/R is equivalent to multiplying 
by G and dividing by R. The same applies for G/B.}), of respective main signal values at each of 
corresponding locations on coordinates in the main scene captured by the image capturing 
devices (see Hashimoto et al, Figure 3; {Main scene signals follow path to the input of Al and 
A2 (i.e. base of the triangle representing the amplifiers.)}), whereby obtaining corrected signal 
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values as corrected values of the main signal value (see Hashimoto et al., Figure 3, R' and B' 
entering the color reproducing correcting circuit). The claim differs from Hashimoto et al. in that 
it further requires a reflection surface for capture of a main scene in the image capturing devices, 
the reflection surface being disposed within a visual field of the camera for reflection of light 
from the main scene or a reference scene disposed near the main scene for reception by the light 
detecting elements via the lens, and the reference signals determined from the reference scene. 

In the same field of endeavor, Tamune teaches an electronic still camera (Figure 1) which 
employs two image capturing devices (Figure 1, image capturing device for photographing "73" 
and image capturing device for scene analysis "86"; {The reference scene is referred to as light 
entering the lens at an angle, and the main scene is light not entering the lens at an angle (i.e. 
light entering on the path shown in Figure 1. Both scenes are reflected by the quick return 
mirror.}). The first image-capturing device is used for photographing a main scene, and the 
second image-capturing device is used for scene analysis (color correction) (Col. 4, Lines 43-52; 
Col. 6, Lines 54-67). Light is sent to the scene analysis image-capturing device via a quick return 
mirror (Figure 1, quick return mirror "71"; {The examiner refers to the reflecting surface as the 
quick return mirror and the lens "301 " of HASHIMOTO et al}\ which reflects the incoming 
light to said image capturing device (Col. 4, Lines 38-47). In light of the teaching of Tamune, it 
would have been obvious to one of ordinary skill in the art to include the quick return mirror and 
second image-capturing device for the white balance calculation in Hashimoto et al., because an 
artisan of ordinary skill in the art would recognize that this would reduce the amount of time 
required for image processing which is implemented by the analyzing the photographic scene 
(see Tamune, Col. 1, Lines 56-60). 
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As to claim 47, Hashimoto et al., as modified by Tamune, teaches the image capturing 
system according to claim 1. Although it is not stated explicitly, Official Notice is taken that 
using reflective plastic mirrors in cameras is well-known and expected in the art. One of ordinary 
skill in the art would recognize that this these mirrors allow can be manufactured at very low 
cost and have excellent transmission properties. Plastic is a material that follows the neutral 
interface reflection theory. See materials cited in Conclusion Section (Non-Patent Literature). 

As to claims 49 and 50, the limitations of claim 49 and 50 can be found in claim 47. 
Therefore, claims 49 and 50 are rejected as previously discussed with respect to claim 47. 

8. Claims 14,46 and 48 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Hashimoto et al. (see Patent Number above) in view of Tamune (see Patent Number above) and 
further in view of Leideg et al. (US # 5,822,625). 

As to claim 14, Hashimoto et al., as modified by Tamune, teaches the image capturing 
system according to claim 1 (see rejection of claim 1 above), wherein each of the image 
capturing devices and the light detecting element is constituted by an individual element of a 
same characteristic (see claim 19 above), the lens being provided individually for each of the 
image capturing device and the light detecting element (see Hashimoto et al., Figure 3, lens 
"301"), the angle and coordinate positions of a starting point of the reflection surface being 
changed continuously in accordance with the focal length of the lens (see claim 25 above), the 
reflection surface being fixed within a maximum visual field of the lens for selection from a 
reference image portion, of selected reference portions corresponding to the reflection surfaces in 
accordance with the focal length (see Tamune, Figure 1, quick return mirror "71" picks up both 
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main scene and reference scene in the interpretation of the examiner). The claim differs from 
Hashimoto et al., as modified by Tamune, in that it further requires that the lenses be 
synchronized in zooming and iris controls. 

In the same field of endeavor, Leideg et al. teaches an electronic camera which captures 
an image on photographic film and captures an image on a low resolution image sensor. The 
zoom lenses for the film and the CCD image sensor are synchronized such that an image is 
viewed through the viewfinder (image on CCD image sensor), as it will be seen on the film 
(Figure 1 ; Col. 3, Lines 50-57; {It is inherent that the apertures of the film and the image sensor 
(Figure 1) are synchronized, because the image is seen on the viewfinder as it will be seen on the 
film,}). In light of the teaching of Leideg et al., it would have been obvious to one of ordinary 
skill in the art to include the synchronized zooming and iris controls of Leideg et al. in the 
system of Hashimoto et al., as modified by Tamune, because an artisan of ordinary skill in the art 
would recognize that this would provide a more faithful calculation of the white balance values 
used to correct the main image of Hashimoto et al, as modified by Tamune. 

As to claim 46, the limitations of claim 46 can be found in claim 14. Therefore, claim 46 
is analyzed and rejected as previously discussed with respect to claim 14. 

As to claim 48, see Official Notice statement of claim 47 above in regard to plastic 
mirrors. 

9. Claim 18 is rejected under 35 U.S.C. 103(a) as being unpatentable over Hashimoto et al. 
(see Patent Number above) in view of Tamune (see Patent Number above) and further in view of 
Ellenby et al. (US # 5,682,332). 
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As to claim 18, Hashimoto et al., as modified by Tamune, teaches the image capturing 
system according to claim 2. The claim differs from Hashimoto et al, as modified by Tamune, in 
that it further requires a CG image generating portion for generation of a computer image and a 
CG light source determining portion for determining a light source color for the computer image 
for virtual multiplication of the corrected signal values in each of the color channels with a light 
source color value obtained by the CG light source determining portion for obtaining a secondary 
corrected image, the secondary corrected image being merged with the computer image 
generated by the CG image generating portion into a synthesized image. 

In the same field of endeavor, Ellenby et al teaches an image capturing system (Figure 4) 
comprising a computer (Figure 4, computer) with an image generating section (Figure 4, image 
generator "46"), which in accordance with position, attitude, lighting conditions generates an 
image (Col. 6, Lines 29-33; Lines 54-62). The computer connected to a camera and further 
comprising a combining section for combining the camera image with the CGI for output to a 
display (Col. 6, Lines 28-35). In light of the teaching of Ellenby et aL, it would have been 
obvious to one of ordinary skill in the art to include computer of Ellenby et al. to the camera of 
Hashimoto et al., as modified by Tamune, for composite image display, because an artisan of 
ordinary skill in the art would recognize that this would provide a system with highly 
sophisticated image processing manipulations of a real scene (see Ellenby et al, Col. 3, Lines 6- 
8). 
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10. Claims 33-35 are rejected under 35 U.S.C. 103(a) as being unpatentable over Hashimoto 
et al. (see Patent Number above) in view of Tamune (see Patent Number above) and further in 
view of Cooper (US # 6,215,962). 

As to claim 33, Hashimoto et al., as modified by Tamune, teaches the image capturing 
system according to claim 2. The claim differs from Hashimoto et al., as modified by Tamune, in 
that it further requires that the correction unit includes means for measuring a complimentary 
color of a color determined by the reference signal values, and optical filter means including an 
optical filter for reproducing the complementary color and altering a color of an image which 
reaches the image capturing devices. 

In the same field of endeavor, Cooper teaches a digital camera (Figure 1), which uses a 
plurality of band-pass filters to determine an illumination condition from the colors of an image, 
and from this information, it is determined how the image is to be altered to compensate for color 
differences (Col. 4, Lines 24-67). In light of the teaching of Cooper, it would have been obvious 
to one of ordinary skill in the art to include the band-pass filter arrangement of Cooper in the 
image sensor of Hashimoto et al., because an artisan of ordinary skill in the art would recognize 
that this can effectively compensate for color differences in an image (see Cooper, Col. 3, Lines 
11-15) allowing for a more natural picture. 

As to claim 34, Hashimoto et al., as modified by Tamune and Cooper, teaches the image 
capturing system according to claim 33, wherein the optical filter is disposed so as to alter a 
color of the image which reaches the light detecting elements (see Cooper, Figure 3, band-pass 
filters "1 16A-E"; {The filters alter whether the color is detected by the photodiode.}), the means 
for obtaining the complementary color controlling the optical filter so as to bring the color 
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balance of the reference signal values as close as possible to a required color balance (see 
Cooper, Col. 2, Lines 42-62; Col. 5, Lines 28-50; "...proper selection of band-pass filters..."). 

As to claim 35, Hashimoto et al., as modified by Tamune, teaches the image capturing 
system according to claim 33, wherein the optical filter means includes a plurality of preset 
filters each having a color balance different from the others, one of the present filters closest to 
the complementary color being selected (see Cooper, Col. 4, Lines 50-67). 

Conclusion 

1 1 . The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. In the Abstract of the IEEE Explorer Article (see Notice of References), it is shown 
that plastics follow the neutral interface reflection theory. Thus a plastic mirror would be a 
surface that follows this theory. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Anthony J. Daniels whose telephone number is (571) 272-7362. 
The examiner can normally be reached on 8:00 A.M. - 5:30 P.M.. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ngoc-Yen Vu can be reached on (571) 272-7320. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



AD 

5/11/2006 




